我们分析了牛顿方法的变体的性能，并通过二次正则化来解决复合凸最小化问题。在我们方法的每个步骤中，我们选择正规化参数与当前点的梯度标准的某些功率成正比。我们介绍了一个以h \ h \“第二或第三个衍生物的较旧连续性为特征的问题类别。然后，我们使用简单的自适应搜索步骤介绍该方法，允许自动调整问题类，并以最佳的全球复杂性界限，而无需知道问题的特定参数。特别是，对于Lipschitz连续第三个导数的函数类别，我们获得了全局$ o（1/k^3）$ rate，以前归因于三阶张量方法。功能是均匀凸的，我们证明我们方案的自动加速度是合理的，导致全局速率和局部超线性收敛。不同的速率（sublinear，linear和superlinear）之间的切换是自动的。同样，没有先验的先验需要了解参数。

在这项工作中，我们提出了新的自适应步长策略，以改善几种随机梯度方法。我们的第一种方法（停止）基于经典的Polyak步长（Polyak，1987），是随机优化SPS（Loizou等，2021）的最新开发的延伸，我们的第二种方法，以及我们的第二种方法表示毕业生，通过“随机梯度的多样性”重新缩放步长。我们对这些方法进行了理论分析，以实现强烈凸平的光滑功能，并表明尽管随机梯度随机梯度，它们仍享有确定性的速率。此外，我们证明了自适应方法对二次目标的理论优势。不幸的是，两个停止和毕业生都取决于未知数量，这仅适用于过度散光模型。为了解决这个问题，我们放弃了这种不希望的依赖性，并重新定义了停止和毕业生的停止和毕业。我们表明，这些新方法在相同的假设下线性收敛到最佳解决方案的邻域。最后，我们通过实验验证来证实我们的理论主张，这表明GRAD对于深度学习优化特别有用。

当任何延迟较大时，异步随机梯度下降（SGD）的现有分析显着降低，给人的印象是性能主要取决于延迟。相反，无论梯度中的延迟如何，我们都证明，我们可以更好地保证相同的异步SGD算法，而不是仅取决于用于实现算法的平行设备的数量。我们的保证严格比现有分析要好，我们还认为，异步SGD在我们考虑的设置中优于同步Minibatch SGD。为了进行分析，我们介绍了基于“虚拟迭代”和延迟自适应步骤的新颖递归，这使我们能够为凸面和非凸面目标得出最先进的保证。

我们介绍了一种牛顿型方法，可以从任何初始化和带有Lipschitz Hessians的任意凸面目标收敛。通过将立方规范化与某种自适应levenberg - Marquardt罚款合并来实现这一目标。特别地，我们表明由$ x ^ {k + 1} = x ^ k - \ bigl（\ nabla ^ 2 f（x ^ k）+ \ sqrt {h \ | \ nabla f（x ^ k）给出的迭代）\ |} \ mathbf {i} \ bigr）^ { - 1} \ nabla f（x ^ k）$，其中$ h> 0 $是一个常数，用$ \ mathcal {o}全球收敛（\ frac{1} {k ^ 2}）$率。我们的方法是牛顿方法的第一个变体，具有廉价迭代和可怕的全球融合。此外，我们证明当目的强烈凸起时，本地我们的方法会收敛超连续。为了提高方法的性能，我们提供了一种不需要超参数的线路搜索程序，并且可提供高效。

我们考虑最小化三个凸功能的总和，其中第一个f是光滑的，第二个f是非平滑且可近的，第三个是与线性操作员L的非光滑近似函数的组成。此模板问题具有许多应用程序，有许多应用程序，有许多应用程序，，具有许多应用程序，，具有许多应用程序。例如，在图像处理和机器学习中。首先，我们为这个问题提出了一种新的原始偶算法，我们称之为PDDY。它是通过将davis-yin分裂应用于原始二重式产品空间中的单调包含的，在特定度量下，操作员在特定度量下是单调的。我们显示了三种现有算法（Condat-VU算法的两种形式） PD3O算法）具有相同的结构，因此PDDY是这种自洽的原始偶算法中的第四个丢失链接。这种表示可以简化收敛分析：它使我们能够总体上得出sublinear收敛速率，而线性收敛导致存在强凸度的存在。此外，在我们的广泛而灵活的分析框架内，我们提出了对算法的新随机概括，其中使用了Friancation降低F梯度的随机估计值，而不是真实的梯度。此外，我们作为pddy的特殊情况获得了线性收敛算法，用于在线性约束下最小化强凸功能f。我们讨论了其对分散优化的重要应用。

We provide a new analysis of local SGD, removing unnecessary assumptions and elaborating on the difference between two data regimes: identical and heterogeneous. In both cases, we improve the existing theory and provide values of the optimal stepsize and optimal number of local iterations. Our bounds are based on a new notion of variance that is specific to local SGD methods with different data. The tightness of our results is guaranteed by recovering known statements when we plug H " 1, where H is the number of local steps. The empirical evidence further validates the severe impact of data heterogeneity on the performance of local SGD.

This paper addresses the kinodynamic motion planning for non-holonomic robots in dynamic environments with both static and dynamic obstacles -- a challenging problem that lacks a universal solution yet. One of the promising approaches to solve it is decomposing the problem into the smaller sub problems and combining the local solutions into the global one. The crux of any planning method for non-holonomic robots is the generation of motion primitives that generates solutions to local planning sub-problems. In this work we introduce a novel learnable steering function (policy), which takes into account kinodynamic constraints of the robot and both static and dynamic obstacles. This policy is efficiently trained via the policy optimization. Empirically, we show that our steering function generalizes well to unseen problems. We then plug in the trained policy into the sampling-based and lattice-based planners, and evaluate the resultant POLAMP algorithm (Policy Optimization that Learns Adaptive Motion Primitives) in a range of challenging setups that involve a car-like robot operating in the obstacle-rich parking-lot environments. We show that POLAMP is able to plan collision-free kinodynamic trajectories with success rates higher than 92%, when 50 simultaneously moving obstacles populate the environment showing better performance than the state-of-the-art competitors.

Consensus clustering aggregates partitions in order to find a better fit by reconciling clustering results from different sources/executions. In practice, there exist noise and outliers in clustering task, which, however, may significantly degrade the performance. To address this issue, we propose a novel algorithm -- robust consensus clustering that can find common ground truth among experts' opinions, which tends to be minimally affected by the bias caused by the outliers. In particular, we formalize the robust consensus clustering problem as a constraint optimization problem, and then derive an effective algorithm upon alternating direction method of multipliers (ADMM) with rigorous convergence guarantee. Our method outperforms the baselines on benchmarks. We apply the proposed method to the real-world advertising campaign segmentation and forecasting tasks using the proposed consensus clustering results based on the similarity computed via Kolmogorov-Smirnov Statistics. The accurate clustering result is helpful for building the advertiser profiles so as to perform the forecasting.

In computational advertising, a challenging problem is how to recommend the bid for advertisers to achieve the best return on investment (ROI) given budget constraint. This paper presents a bid recommendation scenario that discovers the concavity changes in click prediction curves. The recommended bid is derived based on the turning point from significant increase (i.e. concave downward) to slow increase (convex upward). Parametric learning based method is applied by solving the corresponding constraint optimization problem. Empirical studies on real-world advertising scenarios clearly demonstrate the performance gains for business metrics (including revenue increase, click increase and advertiser ROI increase).

In cost-per-click (CPC) or cost-per-impression (CPM) advertising campaigns, advertisers always run the risk of spending the budget without getting enough conversions. Moreover, the bidding on advertising inventory has few connections with propensity one that can reach to target cost-per-acquisition (tCPA) goals. To address this problem, this paper presents a bid optimization scenario to achieve the desired tCPA goals for advertisers. In particular, we build the optimization engine to make a decision by solving the rigorously formalized constrained optimization problem, which leverages the bid landscape model learned from rich historical auction data using non-parametric learning. The proposed model can naturally recommend the bid that meets the advertisers' expectations by making inference over advertisers' historical auction behaviors, which essentially deals with the data challenges commonly faced by bid landscape modeling: incomplete logs in auctions, and uncertainty due to the variation and fluctuations in advertising bidding behaviors. The bid optimization model outperforms the baseline methods on real-world campaigns, and has been applied into a wide range of scenarios for performance improvement and revenue liftup.